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I .  Background 

This  is  the  final  summary  report  of  a  large-scale  study  of  peer 
ratings  completed  in  1955  at  the  U.  S.  Naval  School,  Officer  Candidate 
(ocs)  in  Newport,  Rhode  Island,  with  the  23  sections  comprising  OCS 
class  23.^  The  study's  objective  was  to  yield  supportive  data  on  the 
use  of  peer  ratings  —  specifically,  "peer  nominations"  —  for  supple¬ 
mental  screening  in  the  OCS. 

The  effects  on  the  reliability  and  validity  of  ratings  were  studied 
for  four  core  factors:  the  length  of  time  the  group  had  been  together; 
the  use  of  a  "research"  set  vs.  a  "real"  set  on  the  peer  rating  forms; 
the  nature  of  the  quality  to  be  rated;  and,  friendship  choice. 

Four  forms,  each  requiring  five  "high"  and  five  ''low"  peer  nomina¬ 
tion^  were  utilized:  a  primary  form,  for  all  sectio)  s,  based  on  "success 
as  a  future  Naval  Officer"  (FO);  and,  by  section,  one  of  three  secondary 
forms,  i.e.,  "leadership  qualities"  (LQ),  "interest  in  and  enthusiasm  for 
the  Naval  Service"  (IE),  and  "probability  of  success  in  OCS"  (OC).  This 
design  was  used  at  the  orientation,  third,  and  sixth  weeks  of  training; 
at  the  thirteenth  week,  only  the  FO  form  was  administered. 

Approximately  half  the  sections  received  a  "research"  set  with 
assurance  that  the  ratings  would  be  used  for  research  purposes  only  and 
would  not  affect  their  Navy  career.  The  other  sections  were  given  an 
"administrative"  set  with  instructions  indicating  that  the  ratings  might 
be  used  administratively.  At  the  close  of  every  administration,  each 
subject  was  asked  to  list  five  people  in  his  section  whom  he  regarded  as 
friends;  a  "research"  set  was  always  used  for  this  form. 

^It  is  a  sincere  pleasure  to  acknowledge  the  splendid  cooperation  of  the 
staff  and  student  personnel  of  the  Naval  OCS  in  the  pursuit  of  this  study. 
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II.  Major  Findings 

Three  technical  reports  have  been  produced  under  this  contract 
(l,  2,  3).  They  have  dealt,  in  order,  with  reliability,  validity  against 
in-training  criteria,  and  the  effects  of  friendship  choice.  Their  major 
findings  are  summarized,  by  topic,  below: 

A.  Reliability 

1.  The  reliability  of  forms  administered  very  early  in  train¬ 
ing — after  the  groups  had  been  together  for  four  to  five  days — was 
a  reasonable  approximation  of  the  reliability  obtained  with  the 
same  forms  and  the  same  groups  at  later  points  in  training. 

2.  The  peer  nomination  scores  secured  at  the  end  of  the  third 
week  of  training  correlated  at  a  high  level  with  those  scores  se¬ 
cured  from  the  same  groups  at  a  later  level,  i.e.,  sixth  week. 

3.  There  was  no  significant  difference  in  the  single-stage 
reliability  or  longitudinal  reliability  of  comparable  forms  admin¬ 
istered  under  the  "research"  as  against  the  "administrative"  set. 

4.  All  forms — irrespective  of  the  four  different  rated  quali¬ 
ties  involved — showed  a  tendency  to  begin  with  substantial  reliability 
and  rise  in  subsequent  administrations  to  only  a  slightly  higher 
plateau. 

B.  In-Training  Validity 

1.  Depending  upon  the  criterion  utilized,  different  forms 
yielded  differential  validity  in  prediction. 

2.  All  forms  showed  a  tendency  to  increase  in  their  correlation 
with  academic  performance  from  stage  to  stage. 

3.  Irrespective  of  the  stage  at  which  administered,  all  forms 
showed  a  generalized  significance  in  the  prediction  of  the  military 
aptitude  grade  assigned  by  superiors. 

4.  The  form  which  best  predicted  the  pass-fail  and  academic 
criteria  was  the  one  requiring  nominations  on  "probability  of  success 
in  OCS";  the  significance  of  this  prediction  was  found  at  every  stage 
of  administration  and  for  both  sets. 

5.  While  different  forms  manifested  significant  differences  in 
validity  against  various  criteria,  there  was  no  general  disparity 
between  the  validity  of  forms  administered  under  the  "research"  as 
against  the  "administrative"  set,  although  certain  interactions  were 
found. 
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C .  Friendship  Choice 

1.  An  average  of  about  two  out  of  five  acknowledged  friends 
were  nominated  "high"  on  peer  ratings;  significantly  fewer  friends 
were  nominated  "high"  at  later  stages  of  training  than  were  nomina¬ 
ted  "high"  at  the  outset  of  training. 

2.  In  general,  significantly  fewer  friends  were  nominated 
"high"  on  the  forms  having  to  do  with  "interest  in  and  enthusiasm 
for  the  Naval  Service"  (IE)  and  "probability  of  success  in  OCS" 

(OC)  than  on  the  forms  dealing  with  "success  as  a  future  officer" 

(FO)  or  "leadership  qualities"  (LQ) . 

3.  A  "friendship  score"  based  upon  the  number  of  friendship 
choices  received  by  a  subject  was  found  to  be  significantly  corre¬ 
lated  with  peer  nomination  scores,  but  not  systematically  related 
to  the  academic  performance  criterion.  Different  forms  yielded 
different  relationships  with  friendship — once  again  IE  and  OC  were 
significantly  lower  than  FO  and  LQ. 

4.  These  relationships  were  not  significantly  different  for 
the  "research"  set  as  against  the  "administrative"  set. 

5.  When  validity  coefficients  for  peer  nomination  scores  were 
corrected  by  partialing  friendship,  it  was  found  that  they  regularly 
retained  their  level  of  magnitude,  thus  demonstrating  that  the 
validity  of  peer  ratings — at  least  for  this  criterion — was  not  unduly 
altered  by  a  "popularity"  factor. 

III.  Conclusions  and  Recommendations 


In  general  terms,  these  findings  support  the  administrative  use  of 
peer  ratings  by  adding  further  evidence  of  their  reliability  and  validity. 
Specifically,  the  data  indicate  that  a  peer  nomination  administered  as 
early  as  the  third  week  of  training  yields  substantially  the  same  informa¬ 
tion  as  that  obtained  at  the  sixth  week,  or  even  later.  Considering  the 
progressive  academic  contamination  of  ratings,  over  time,  this  has  particular 
implications  for  good  prediction. 

In  contrast  to  the  view  that  peer  ratings  constitute  a  "popularity 
contest,"  the  data  indicate  that  they  yield  prediction  of  a  performance 
criterion  without  adverse  effects  from  friendship  ties.  It  may  be  suggested, 
too,  that  this  relationship  operates  so  as  to  favor  as  friends  those  of 


high  status  on  certain  other  continua — i.e.,  "success  as  a  future  officer" — 
rather  than  to  simply  create  high  status  for  friends. 

As  a  practical  matter,  there  is  now  a  reasonable  basis  for  the  appli¬ 
cation  of  early  peer  ratings  to  yield  supplemental  data  for  screening.  The 
particular  merit  of  the  various  forms  has  been  considered  in  detail  in  the 
technical  reports  (l,  2,  3);  in  brief  terms,  however,  it  may  be  recommended 
that  a  precise  form  like  OG,  plus  a  broader  form  like  FO  or  LQ,  be  utilized 
administratively  during  the  first  quarter  of  the  0C3  training  cycle.  Added 
to  this  is  the  fact  that  the  "administrative"  set  leads  to  neither  more 
nor  less  reliable  or  valid  scores  than  those  secured  through  the  presumably 
less  threatening,  or  more  lightly- taken,  "research"  set. 

Although  these  findings  do  illuminate  paths  of  application  it  is 
essential  that  additional  data  be  obtained  for  the  construction  of  a  post¬ 
training  criterion  against  which  forms  may  be  further  validated. 
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